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DETAILED ACTION 
Response to Arguments 

1 . Applicant's arguments, see p.7, filed 2/18/2009, with respect to the objection to 
the specification have been fully considered and are persuasive. The objection of the 
specification has been withdrawn. 

The amendment of claim 17 renders the objection moot. 

2. Applicant's arguments with respect to claims 16 and 17 have been considered 
but are moot in view of the new ground(s) of rejection. 

The amendment of claims 16 and 17 renders the previous 35 U.S.C. 101 
rejections for these claims moot. 

3. Applicant's arguments with respect to claims 4 and 16 have been considered but 
are moot in view of the new ground(s) of rejection. 

The amendment to claims 4 and 16 render the previous 35 U.S.C. 112 second 
paragraph rejections of these claims moot. 

4. Applicant's arguments, see p.8, filed 2/18/2009, with respect to the 35 
U.S.C. 1 12 first paragraph rejection of claim 16 have been fully considered and are 
persuasive. The prior 112 first paragraph rejection of claim 16 has been withdrawn. 

5. Applicant's arguments filed 2/1 8/2009 have been fully considered but they are 
not persuasive. 

Regarding claims 1, 15, 16 and 17, the applicant argues that Numagami does not 
teach computing cost values, that comprise number and extent of transitions in pixel 
luminance values along a path, where the path is related to a spatial disposition of 
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objects in tlie image, wliere computing the cost value for a pixel is performed by 
combining values of pairs of pixels at transitions. Numagami however, as explained in 
the rejection, teaches determining depth by integrating the slopes along a path between 
a pixel in an isodensity line, and another pixel in an area between two consecutive 
isodensitiy lines; see section IV. Since, as admitted by the applicant an isodensity line is 
a line that connects equal intensity pixels, the area between two isodensity lines is a 
transition in value. Further Numagami teaches measuring the slope along the path, 
where the slope is measured by difference in intensity values between the pixels in the 
path, and therefore, since the values are discrete, the change in intensity value between 
pairs of pixels along that line, and further, since depth is calculated by integrating these 
slopes, and again because the values are discrete, inherently requiring measuring the 
number of transitions in values measured, and the extent, that is distance between the 
values; see section IV. Finally, Numagami clearly teaches that the path is calculated by 
calculating the normal of the isodensity line, where this normal is the normal of the 
object at that line, and therefore, Numagami teaches that the path is related to a spatial 
disposition of the object, that is, the normal of the object at that point; see section IV. 
The 35 use 102 rejection for these claims is therefore maintained. 

Regarding the claims 5, 7, 1 1 and 14, the applicant only argues that they are 
allowable in virtue of their dependence from claims 1 . Since, as mentioned above, claim 
1 stands rejected, so do these claims. 
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Claim Objections 

6. Claim 2 is objected to because of tine following informalities: Claim 2 recites the 
expression "consistion" instead of consisting. Appropriate correction is required. 

Claim Rejections - 35 USC § 102 

7. The following Is a quotation of the appropriate paragraphs of 35 U.S.C. 1 02 that 
form the basis for the rejections under this section made in this Office action: 

A person shall be entitled to a patent unless - 

(b) the invention was patented or described in a printed publication in this or a foreign country or in public 
use or on sale in this country, more than one year prior to the date of application for patent in the United 
States. 

8. Claims 1-4, 6, 8-10, 12, 13, and 15-17 are rejected under 35 U.S.C. 102(b) as 
being anticipated by Numagami et al. (Numagami, Y.; Kajiwara, Y.; Nakamura, O.; 
Minami, T., Reconstruction of the 3-D shape of an object from a 2-D intensity image, 
Canadian Conference on Electrical and Computer Engineering, 1995. Volume: 2, 5-8 
Sep 1995 Pages: 1188-1191, provided by applicant, "Numagami" hereinafter). 

Regarding claims 1,15, and 17, Numagami teaches a "method of generating a 
depth map (122) comprising depth values representing distances to a viewer, for 
respective pixels of an image (100)" (i.e., a method to reconstruct 3-D shape form a 2-D 
image, 1®' par. of the Introduction, by calculating the heights of pixels, step 2 of section 
IV), "A depth map generating unit (401) for generating a depth map (122)" (i.e., a 
computer, see section V, par. 9), and a "computer program product" (inherently, since 
the method is performed by a computer) for "computing cost values that comprise 
respective measures of a number of and extent of transitions in luminance and/or color 
and/or color components for pixels of the image on a path related to a spatial disposition 
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of objects in tlie image, wlierein computing a cost value for a first one of the pixels (1 08) 
of the image by combining differences between values of pixels which are disposed on 
a path (1 1 2) from the first one of the pixels (1 08) to a second one of the pixels (1 1 0) 
wherein the second one of the pixels belongs to a predetermined subset of the pixels of 
the image" (i.e., to determine the height of a pixel from a fiducial pixel, where the fiducial 
pixel belongs to an isodensity line of pixels by combining the values of the pixels along 
a reconstruction path, section IV, steps 1-3; determining depth by integrating the slopes 
along a path between a pixel in an isodensity line, and another pixel in an area between 
two consecutive isodensitiy lines; see section IV. Since, as admitted by the applicant an 
isodensity line is a line that connects equal intensity pixels, the area between two 
isodensity lines is a transition in value. Further Numagami teaches measuring the slope 
along the path, where the slope is measured by difference in intensity values between 
the pixels in the path, and therefore, since the values are discrete, the change in 
intensity value between pairs of pixels along that line, and further, since depth is 
calculated by integrating these slopes, and again because the values are discrete, 
inherently requiring measuring the number of transitions in values measured, and the 
extent, that is distance between the values; see section IV. Finally, Numagami clearly 
teaches that the path is calculated by calculating the normal of the isodensity line, 
where this normal is the normal of the object at that line, and therefore, Numagami 
teaches that the path is related to a spatial disposition of the object, that is, the normal 
of the object at that point; see section IV); "and assigning a first one of the depth values 
corresponding to the first one of the pixels (108) on basis of the cost value" (i.e.. 
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assigning tlie lieiglit of tlie pixel by integrating the slope values of the pixels along the 
path, where the slope value is dependent on the intensity level of the pixels, section IV, 
step 2 (ill)). 

Regarding claim 2, Numagami further teaches that "the predetermined subset 
comprises one selected from the group consistion of (!) pixels which are located at a 
border of the image, (ii) pixels of a part of the border, and (iii) a central pixel of the 
image" (i.e., the predetermined subset of pixels belongs to an isodensity line of pixels, 
and this line is not restricted to pixels outside the border of the image, or the central 
portion of the image; see section III, subsection Extraction of the isodensity lines, 1st 
par.). 

Regarding claim 3, Numagami also discloses that "a first one of the differences is 
equal to a difference between respective values of neighboring pixels which are 
disposed on the path (112)" (i.e., the slope value of the pixels along the path, where the 
slop value is dependent on the difference of intensity between the pixels in the path, 
section IV, step 2 (iii)). 

Regarding claim 6, Numagami further discloses that "the cost value for the first 
one of the pixels (108) is computed by accumulating the differences between the values 
of the pixels which are disposed on the path (112)" (i.e., the cost is calculated by 
integrating the slope value along the path, where the slope value depends on the 
difference in intensity value between the pixels in the path, section IV, step 2 (iii)). 

Regarding claim 8, Numagami also teaches that "the cost value for the first one 
of pixels is computed by accumulating products of differences between the values of the 
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pixels wliicli are disposed on tlie patli (112) and respective weighting factors for tlie 
differences" (i.e., the slope value is compensated by a product of a weight factor, see 
section IV, subsection Calculation of the height, 3"^ par., and formula 12). 

Regarding claim 9, Numagami further teaches that "a first one of the weighting 
factors which is related to a difference between a value of a particular pixel and a value 
of its neighboring pixel, is based on a distance between the particular pixel and the first 
one of the pixels (108)" (see section IV, subsection Calculation of the height, 3rd par., 
formula 12, and Fig. 5). 

Regarding claim 10, Numagami also discloses that "whereby a second one of the 
weighting factors which is related to a difference between a value of a particular pixel 
and a value of its neighboring pixel, is based on the location of the neighboring pixel 
related to the particular pixel" (i.e., the weight factor is based on the angle between the 
pixels, see section IV, subsection Calculation of the height, 3rd par., formula 12, and 
Fig. 5). 

Regarding claim 12, Numagami also teaches "computing a second cost value for 
a third one of the pixels on basis of the cost value for the first one of the pixels" (i.e., 
linear interpolation used to fill areas between reconstruction paths, see section V, par. 7 
and 8, and fig. 9). 

Regarding claim 13, Numagami further teaches "computing the second cost 
value by combining the cost value of the first one of the pixels with a difference between 
further values of further pixels which are disposed on a second path from the third one 
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of the pixels to the first one of the pixels" (i.e., linear interpolation used to fill areas 
between reconstruction paths, see section V, par. 7 and 8, and fig. 9). 

Regarding claim 16, Numagami teaches an "image processing apparatus" (i.e., a 
computer, see section V, par. 9), "comprising: receiving means (502) for receiving a 
signal corresponding to an image (100)" (inherently, since the computer processes a 2- 
D image, which must have been received) "and a depth generating unit (401) for 
generating a depth map (122), wherein said map generating unit includes computing 
means for computing cost values that comprise respective measures of a number of 
and extent of transitions in luminance and/or color and/or color components for pixels of 
the image on a path related to a spatial disposition of objects in the image, wherein 
computing includes computing a cost value for a first one of the pixels (108) of the 
image by combining differences between values of pairs of neighboring connected 
pixels at transitions which are disposed on a path (1 1 2) from the first one of the pixels 
(108) to a second one of the pixels (110), wherein the second one of the pixels belongs 
to a predetermined subset of the pixels of the image, and assigning means for assigning 
a first one of the depth values corresponding to the first one of the pixels (1 08) on basis 
of the cost value" (see rejection for claim 1 , above). 

Claim Rejections - 35 USC § 103 
9. The following is a quotation of 35 U.S.C. 1 03(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
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invention was made to a person liaving ordinary skill in tlie art to wliicli said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

10. Claims 4, 5 and 7 are rejected under 35 U.S.C. 103(a) as being unpatentable 
over Numagami et al. (Numagami, Y.; Kajiwara, Y.; Nakamura, O.; Minami, T., 
Reconstruction of the 3-D shape of an object from a 2-D intensity image, Canadian 
Conference on Electrical and Computer Engineering, 1995. Volume: 2, 5-8 Sep 1995 
Pages: 1188-1191, provided by applicant, "Numagami" hereinafter) as applied to claim 
1 above, and further in view of Cahill et al. (U.S. Patent Publication No. 2004/0062439, 
"Cahill" hereinafter). 

Regarding claim 4, Numagami does not explicitly teach that "a second one of the 
differences is equal to an absolute value of difference between respective values of 
neighboring pixels which are disposed on the path (112)". Cahill also teaches a method 
of providing depth information from a 2D image, comprising calculating a depth map, 
see par. [0002], in which values are filtered for noise by using a threshold, see par. 
[0022]. Because both Numagami and Cahill disclose ways of obtaining 3D information 
from a 2D image, it would have been obvious to one of ordinary skill in the art at the 
time of the invention to apply the threshold correction to the measurement of intensity 
difference between pixels disclosed in Numagami. Such a modification would allow the 
method in Numagami not to produce erroneous depth estimations when a noisy value is 
encountered, which could result in a relatively flat surface appearing discontinuous in its 
3D reconstruction. Obviously, when applying a threshold to the difference in intensity 
value, the absolute value would be used, since the noisy value could cause both an 
erroneous negative or positive value. 
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Regarding claim 5, Numagami also teaches that "the values of pixels correspond 
to one of luminance (....)" (i.e., the value of the pixel is the intensity of the pixel, section 
IV, step 2 (iii)). Numagami does not teach that the values of the pixels are measured in 
terms of color. Cahill however, also teaches a method of generating a depth map, see 
rejection for claim 4 above, and par. [0002] for color used in conjunction with luminance 
for segmentation. 

Because both Cahill and Numagami teach methods of creating depth maps from 
a 2-D image, it would have been obvious to one of ordinary skill in the art at the time of 
the invention to modify the pixel evaluation of luminance in Numagami with the 
alternative evaluation of chrominance taught in Cahill because in luminance 
measurements, the level of white for every pixel is measured. In chrominance 
measurements, the levels of the color components are measured. These techniques are 
equivalent, one applying to grey-level pictures, and the other to color pictures, and well- 
known in the art. 

Regarding claim 7, Numagami discloses that "the cost value for the first one of 
the pixels (108) is computed by accumulating the differences between the values of the 
pixels which are disposed on the path (112)" (i.e., the cost is calculated by integrating 
the slope value along the path, where the slope value depends on the difference in 
intensity value between the pixels in the path, section IV, step 2 (iii)). Numagami does 
not teach "the differences being larger than a predetermined threshold". Cahill however, 
also teaches a method of generating a depth map, in which the pixel values are 
adjusted to meet a predetermined threshold, see discussion for claim 4 above. 
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1 1 . Claim 1 1 Is rejected under 35 U.S.C. 103(a) as being unpatentable over 
Numagami et al. (Numagami, Y.; Kajiwara, Y.; Nakamura, O.; Minami, T., 
Reconstruction of the 3-D shape of an object from a 2-D intensity image, Canadian 
Conference on Electrical and Computer Engineering, 1995. Volume: 2, 5-8 Sep 1995 
Pages: 1188-1191, provided by applicant, "Numagami" hereinafter). 

Regarding claim 1 1 , Numagami teaches "computing a second cost value for the 
first one of the pixels (108) of the image by combining differences between values of 
pixels which are disposed on a second path (202) from the first one of the pixels (108) 
to a third one of the pixels (204) which belongs to the predetermined subset of the 
pixels of the image" (Numagami teaches that its method is performed for every one of 
the fiducial points found, see section IV, step 3, where a fiducial point is a point of an 
Isodenslty line, see section IV, step 1 , and there is a plurality of Isodenslty lines, see fig 
2). Numagami does not teach "determining the minimum of the cost value and the 
second cost value; assigning the first one of the depth values corresponding to the first 
one of the pixels (108) on basis of the minimum". Although Numagami is silent about 
how to resolve conflicting cost values for the same pixel, where the cost values have 
been calculated along the different paths between this pixel and fiducial points In 
adjacent Isodenslty lines, the examiner takes the official notice that, where two or more 
conflicting measurements for a unique value are found, there are a limited number of 
choices on how to solve said conflict. Therefore it would have been obvious for one of 
ordinary skill in the art to modify the method disclosed in Numagami, with a choice of 
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the minimum value when two or more conflicting values are found for the same 
measurement to resolve this conflict by either choosing the larger value, the lower value 
or the average value. Arriving at the conclusion that the minimum value is more likely to 
be correct would have come naturally to one of ordinary skill in the art at the time of the 
invention after normal testing of the method, if, for example, it was found that the most 
frequent error in measurement is to overestimate the value for a point. 

12. Claim 14 is rejected under 35 U.S.C. 103(a) as being unpatentable over 
Numagami et al. (Numagami, Y.; Kajiwara, Y.; Nakamura, O.; Minami, T., 
Reconstruction of the 3-D shape of an object from a 2-D intensity image, Canadian 
Conference on Electrical and Computer Engineering, 1995. Volume: 2, 5-8 Sep 1995 
Pages: 1 188-1 191 , provided by applicant, "Numagami" hereinafter) as applied to claim 
12 above, and further in view of Nakatsuna et al. (U.S. Patent Publication No. 
2002/0154116). 

Regarding claim 14, Numagami also teaches that "cost values corresponding to 
respective pixels of the image are successively computed on basis of further cost 
values being computed for further pixels" (i.e., pixel values are calculated by linear 
Interpolation of neighboring pixels, see section V, 8th par.). Numagami does not teach 
"a first scan direction of successive computations of cost values for a first row of pixels 
of the image being opposite to a second scan direction of successive computations of 
cost values for a second row of pixels of the image". Nakatsuna, however, teaches a 
method of interpolating depth values on a pixel-by-pixel basis (see par. [01 19] and 
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[0165]), in which the pixels are evaluated in a zigzag path, so that pixels may be 
positioned in a two-dimensional neighborhood (see par. [0179]). 

Because both Numagami and Nakatsuna disclose linear interpolation for 
calculating pixel values, it would have been obvious to one of ordinary skill in the art at 
the time of the invention to combine the linear interpolation of depth values as disclosed 
in Numagami, with the zigzag inspection path disclosed in Nakatsuna. Such an 
approach would be representative of the well known principle of locality in program 
optimization, by which it is advantageous to perform tasks in such an order that the 
values that have just been calculated and are therefore readily available, are those 
needed to perform the next calculation. 

Conclusion 

1 3. Applicant's amendment necessitated the new ground(s) of rejection presented in 
this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP 
§ 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 
CFR 1.136(a). 

A shortened statutory period for reply to this final action is set to expire THREE 
MONTHS from the mailing date of this action. In the event a first reply is filed within 
TWO MONTHS of the mailing date of this final action and the advisory action is not 
mailed until after the end of the THREE-MONTH shortened statutory period, then the 
shortened statutory period will expire on the date the advisory action is mailed, and any 
extension fee pursuant to 37 CFR 1 .136(a) will be calculated from the mailing date of 
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the advisory action. In no event, liowever, will the statutory period for reply expire later 
than SIX MONTHS from the date of this final action. 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Carlos Perromat whose telephone number is (571) 270- 
7174. The examiner can normally be reached on M-TH 8:30 am- 5:00 pm EST. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Kee M. Tung can be reached on (571)272-7794. The fax phone number for 
the organization where this application or proceeding is assigned is 571-273-8300. 

Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a 
USPTO Customer Service Representative or access to the automated information 
system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. 

/Kee M Tung/ Carlos Perromat 

Supervisory Patent Examiner, Art Unit 2628 Examiner 

Art Unit 2628 

CP. 
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